Oracle lower bounds for stochastic gradient sampling algorithms
نویسندگان
چکیده
We consider the problem of sampling from a strongly log-concave density in Rd, and prove an information theoretic lower bound on number stochastic gradient queries log needed. Several popular algorithms (including many Markov chain Monte Carlo methods) operate by using gradients to generate sample; our results establish limit for all these algorithms. show that every algorithm, there exists well-conditioned target which distribution points generated algorithm would be at least ε away total variation distance if is less than Ω(σ2d/ε2), where σ2d variance gradient. Our follows combining ideas Le Cam deficiency routinely used comparison statistical experiments along with standard tools bounding Bayes risk functions. To best knowledge provide first nontrivial dimension-dependent this problem.
منابع مشابه
Lower Bounds for Sampling Algorithms for Estimating the Average
We show lower bounds on the number of sample points and on the number of coin tosses used by general sampling algorithms for estimating the average value of functions over a large domain. The bounds depend on the desired precision and on the error probability of the estimate. Our lower bounds match upper bounds established by known algorithms, up to a mul-tiplicative constant. Furthermore, we g...
متن کاملSampling Bounds for Stochastic Optimization
A large class of stochastic optimization problems can be modeled as minimizing an objective function f that depends on a choice of a vector x ∈ X, as well as on a random external parameter ω ∈ Ω given by a probability distribution π. The value of the objective function is a random variable and often the goal is to find an x ∈ X to minimize the expected cost Eω[fω(x)]. Each ω is referred to as a...
متن کاملSequential Sampling Algorithms: Unified Analysis and Lower Bounds
Sequential sampling algorithms have recently attracted interest as a way to design scalable algorithms for Data mining and KDD processes. In this paper, we identify an elementary sequential sampling task (estimation from examples), from which one can derive many other tasks appearing in practice. We present a generic algorithm to solve this task and an analysis of its correctness and running ti...
متن کاملLower Bounds for Heuristic Algorithms
The constraint satisfaction problem, the Boolean satisfiability problem and the graph isomorphism problem do not have efficient algorithms. In order to solve these problems, one utilizes heuristic algorithms of polynomial running time. The present thesis studies three classical heuristics for the above-mentioned decision problems and answers the question whether they can be implemented more eff...
متن کاملLower Bounds for Online Algorithms
What is an oline problem Unlike the offline problems that we get the whole input at the beginning, in an online problem we just have some part of the input in advance. The rest of the input are queries that arrive one by one. Upon the arrival of each request, we need to process as it is received. In analyzing online algorithms, we usually, do not really care about the ruing time. The source of ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Bernoulli
سال: 2022
ISSN: ['1573-9759', '1350-7265']
DOI: https://doi.org/10.3150/21-bej1377